On computational irreducibility and the predictability of complex physical systems 
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Using elementary cellular automata (CA) as an example, we show how to coarse-grain CA in all 
classes of Wolfram's classification. We find that computationally irreducible (C1R) physical processes 
can be predictable and even computationally reducible at a coarse-grained level of description. The 
resulting coarse-grained CA which we construct emulate the large-scale behavior of the original 
systems without accounting for small-scale details. At least one of the CA that can be coarse- 
grained is irreducible and known to be a universal Turing machine. 

PACS numbers: 05.45.Ra, 05.10.Cc, 47.54.+r 



Can one predict the future evolution of a physical pro- 
cess which is described or modeled by a computationally 
irreducible (CIR) mathematical algorithm? For such sys- 
tems, in order to know the system's state after (e.g.) one 
million time steps, there is no faster algorithm than to 
solve the equation of motion a million time steps into the 
future. Wolfram has suggested that the existence of CIR 
systems in nature is at the root of our apparent inability 
to model and understand complex systems 0, 0, 01 Q ■ 

Complex physical systems that are CIR might there- 
fore seem to be inherently unpredictable. It is tempting 
to conclude from this that the enterprise of physics itself 
is doomed from the outset; rather than attempting to 
construct solvable mathematical models of physical pro- 
cesses, computational models should be built, explored 
and empirically analyzed. This argument, however, as- 
sumes that infinite precision is required for the prediction 
of future evolution. Usually coarse-grained or even statis- 
tical information is sufficient: indeed, a physical model 
is usually correct only to a certain level of resolution, 
so that there is little interest in predictions from such a 
model on a scale outside its regime of validity. 

In this Letter, we report on experiments with near- 
est neighbour one-dimensional cellular automata, which 
show that because in practice one only seeks coarse- 
grained information, complex physical systems can be 
predictable and even computationally reducible at some 
level of description. The implication of these results is 
that, at least for systems whose complexity is the out- 
come of very simple rules, useful approximations can be 
made that enable predictions about future behavior. 

Cellular automata (CA) are dynamical systems com- 
posed of a lattice of cells. Each cell in the lattice can 
assume a value from a given finite alphabet. The sys- 
tem evolves in time according to an update rule that 
gives a cell's new state as a function of values in its fi- 
nite neighborhood. CA were originally introduced by von 
Neumann and Ulam [5( in the 1940's as a possible way of 
simulating self reproduction in biological systems. Since 
then, CA have attracted a great deal of interest in physics 
Hi IE 0] because they capture two basic ingredients of 
many physical systems: 1) they evolve according to a lo- 
cal uniform rule. 2) CA can exhibit rich behavior even 



with very simple update rules. For similar and other rea- 
sons, CA have also attracted attention in computer sci- 
ence 0, , biology > material science 0] and other 
fields. 

In early work 0, 0, 0, El , Wolfram proposed that CA 
can be grouped into four classes of complexity. Class 1 
consists of CA whose dynamics reaches a steady state re- 
gardless of the initial conditions. Class 2 consists of CA 
whose long time evolution produces periodic or nested 
structures. CA from both of these classes are simple 
in the sense that their long time evolution can be de- 
duced from running the system a small number of time 
steps. On the other hand, class 3 and class 4 consist 
of "complex" CA. Class 3 CA produce structures that 
seem random. Class 4 CA produce a mixture of random 
structures and periodic behavior. This transition in CA 
complexity was later regarded as a phase transitional 
in the CA rule space. For a review on CA classification 
see Refs.SIl! There is no generally agreed upon al- 
gorithm for classifying a given CA. The assignment of 
CA to these four classes is somewhat subjective and, we 
will argue, may need to be refined. Based on numerical 
experiments, Wolfram hypothesized Q, 0, El that most 
CA from class 3 and 4 are CIR. 

There is no unique way to define coarse-graining, but 
here we will mean that our information about the CA is 
locally coarse-grained in the sense of being stroboscopic in 
time, but that nearby cells are grouped into a supercell 
according to some specified rule (as is frequently done 
in statistical physics). A system which can be coarse- 
grained is compactable since it is possible to calculate its 
future time evolution (or some coarse aspects of it) using 
a more compact algorithm than its native description. 
Note that our use of the term compactable refers to the 
phase space reduction associated with coarse-graining, 
and is agnostic as to whether or not the coarse-grained 
system is reducible or irreducible. Accordingly, we de- 
fine predictable to mean that a system is computation- 
ally reducible or has a computationally reducible coarse- 
graining. Thus, it is possible to calculate the future time 
evolution of a predictable system (or some coarse aspects 
of it) using an algorithm which is more compact than 
both the native and coarse-grained descriptions. 
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In order to quantify the implications of looking at 
coarse-grained information only, we have systematically 
attempted to coarse-grain the 256 nearest neighbor one- 
dimensional binary CA that were the subject of Wol- 
fram's investigations 0, |(|. The outcome, described in 
detail below, is surprising: we found that many CA can 
be coarse-grained and that in some cases CIR CA are 
coarse-grained by computationally reducible ones. In 
other words, even though microscopically a given sys- 
tem might be CIR, its coarse-grained dynamics can be 
compactable and predictable. 

We start by defining a simple procedure for coarse- 
graining a CA. Other constructions are undoubtedly pos- 
sible. For simplicity we limit our treatment to one- 
dimensional systems with nearest neighbor interactions. 
Generalizations to higher dimensions and different inter- 
action radii are straightforward. Let A = (a (t) , S A , /a) 
be a cellular automaton defined on an array of cells 
a(t) = {a n (i)}|^L_oo- Each cell accepts an alphabet 
of Sa symbols, namely a n (t) S {0 . . . Sa — 1}- The val- 
ues of the cells evolve in time according to the update 
rule a n (t + 1) = f A K-i CO i a n (0 , «n+i (*)], where f A ■ 
{Sa} 3 — > {Sa} is the transition function. The update 
rule is applied simultaneously to all the cells and we de- 
note this application by a (t + 1) = f A • a it). 

Our goal is to find a modified CA B — (b (t) , Sg,fg) 
and an irreversible coarse-graining function b — C (a) , 
which are capable of a coarse-grained emulation of A. 
For every initial condition a(0), B and C must satisfy: 



construction is possible only if 



C(/r*-a(0)) =/|-C(a(0)). 



(1) 



Namely, running the original CA for T ■ t time steps and 
then coarse-graining is equivalent to coarse-graining the 
initial condition and then running the modified CA t time 
steps. The constant T is a time scale associated with the 
coarse- graining. 

To search for explicit coarse-graining rules, we de- 
fine the N'th block version A N = (a N , S a nJ a n) of A. 

San — (S A ) N and each cell in A N represents a block of 
N cells in A. Cell values are translated between A and 
A N according to the base Sa value of N cells in A. The 
transition function f A N : {S A ^ } 3 — * {S A N } is computed 
by running A for N time steps on all possible initial con- 
ditions of length 37V. In this way A N computes in one 
time step N time steps of A. Note that A N is not a 
coarse-graining of A because no information was lost in 
the cell translation. 

Next we attempt to generate the coarse CA B by pro- 
jecting the alphabet of A N on a subset of {0 . . . Sa n — 1}- 
This is the key step where information is being lost, a ma- 
nipulation which distinguishes between coarse-graining 
and emulation blocking transformations 0, Q . The tran- 
sition function fg is constructed from f A N by projecting 
its arguments and outcome: 



f A N [xi,x 2 ,x 3 ] = f A N [2/1,2/2,2/3], V(x,y\xi = yi). (3) 

Otherwise, fg is multi-valued and our coarse-graining 
attempt fails for the specific choice of N and P. 

In cases where the above conditions are satisfied, 
the resulting CA B is a coarse-graining of A N with 
a time scale T = 1. For every step (t + 1) = 
f A N [a^ r _ 1 (f), (t), a^ +1 (t)] of A N , B makes the move 



b n {t+l) = f B [bn-i{t),b n (t),b n+1 {t)} 



(4) 



fA N 



<_i(0,^(0,<+iW =<#(* + !), 



fB[xi,X2,X 3 }= f A N [x 1 ,X 2 ,X 3 ]. (2) 

Here x = P (x) denotes the projection operation. This 



where we have used Eq. J3Jl in the last step. B therefore 
satisfies Eq. Q with P as the coarse-graining function. 
Since a single time step of computes N time steps of 
A, B is also a coarse-graining of A with a coarse-grained 
time scale T = N. The coarse-graining function C in 
this case is composed of the translation from A to A N 
followed by the projection operator P. Analogies of these 
operators have been used in attempts to reduce the com- 
putational complexity of certain stochastic partial differ- 
ential equations 0, [H| • Similar ideas have been used to 
calculate critical exponents in probabilistic CA 0, . 

It is interesting to notice that the above coarse-graining 
procedure can lose two very different types of dynamic 
information. To see this, consider Eq. @. This equation 
can be satisfied in two ways. In the first case 

f A N [xi, x 2 ,x 3 ] = f A N [yi , y 2 , 2/3] , V (x, y\x~i = yt) , (5) 

which necessarily leads to Eq. f A N m this case is 
insensitive to the projection of its arguments. The dis- 
tinction between two variables which are identical under 
projection is therefore irrelevant to the dynamics of A N , 
and by construction to the long time dynamics of A. By 
eliminating irrelevant degrees of freedom (DOF), coarse- 
graining of this type removes information which is re- 
dundant on the microscopic scale. The coarse CA in this 
case accounts for all possible long time trajectories of the 
original CA and the complexity classification of the two 
CA is therefore the same. 

In the second case Eq. © is satisfied even though Eq. 
iJSJ is violated. Here the distinction between two vari- 
ables which are identical under projection is relevant to 
the dynamics of A. Replacing x by y in the initial con- 
dition may give rise to a difference in the dynamics of A. 
Moreover, the difference can be (and in many occasions 
is) unbounded in space and time. Coarse-graining in this 
case is possible because the difference is constrained in 
the symbol space by the projection operator. Namely, 
projection of all such different dynamics results in the 
same coarse-grained behavior. Note that the coarse CA 
in this case cannot account for all possible long time tra- 
jectories of the original one. It is therefore possible for 
the original and coarse CA to fall into different complex- 
ity classifications. 
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Coarse-graining by elimination of relevant DOF re- 
moves information which is not redundant with respect 
to the original system. The information becomes redun- 
dant only when moving to the coarse scale. In fact, "re- 
dundant" becomes a subjective qualifier here since it de- 
pends on our choice of coarse description. In other words, 
it depends on what aspects of the microscopic dynamics 
we want the coarse CA to capture. In a sense, this is 
analogous to the subtleties encountered in constructing 
renormalization group transformations for the critical be- 
havior of antiferromagnets [l^. |20| . 

We now give specific examples of coarse-graining CA. 
In the sequel, CA rules are numbered using Wolfram's 
notation^ @- Figure ^ ( a ) an d (b) shows a coarse- 
graining of rule 146 by rule 128. Rule 146 produces a 
complex, seemingly random behavior which falls into the 
class 3 group. We use a super-cell size N = 3, and project 
the alphabet {0 ... 7} of the super-cells back to the {0, 1} 
alphabet with P(7) = 1 and P(x ^ 7) = 21]. A triplet 
of cells in rule 146 are therefore coarse-grained to a single 
cell and the value of the coarse cell is 1 only when the 
triplet are all 1. Using this projection operator we con- 
struct the transition function of the coarse CA which is 
found to be rule 128, a class 1 CA. This choice of coarse- 
graining eliminates the small scale details of rule 146. 
Only structures of lateral size of three or more cells are 
accounted for. The decay of such structure in rule 146 is 
accurately described by rule 128. 

Note that a class 3 CA was coarse-grained to a class 1 
CA in the above example. Our gain was therefore two- 
fold. In addition to the phase space reduction associated 
with coarse-graining we have also achieved a reduction 
in complexity. Our procedure found predictable coarse- 
grained aspects of the dynamics even though the small 
scale behavior of rule 146 is complex, potentially CIR. 
As we explained earlier, this type of simplification can 
be achieved only by eliminating relevant DOF. 

As a second example we show a transition between 
rules with a comparable complexity. Fig. ^ (c) and (d) 
shows a coarse-graining of rule 105 by rule 150. N = 2 
in this example and P(x) = 1 only when x = 0, 3 [2 1| . 

The coarse-graining procedure we described above is 
not constructive, but instead is a self-consistency con- 
dition on a putative coarse-graining rule with a specific 
block size N and projection operator P. In many cases 
the coarse-graining fails and one must try other choices 
of N and P. Can all CA be coarse-grained? If not, which 
CA can be coarse-grained and which cannot? 

To answer these questions we tried systematically to 
coarse-grain Wolfram's 256 elementary rules. We applied 
the coarse-graining procedure to each rule and scaned the 
N,P space for valid solutions. In this way we were able 
to coarse-grain 240 out of the 256 CA These 240 

coarsen-able rules include members of all four classes. 
Many elementary CA can be coarse-grained by other ele- 
mentary CA. Figure[3shows a map of the coarse-graining 
transitions that we found within the family of elementary 
rules. Only coarse-grainings with N < 4 are shown due 




FIG. 1: Examples of coarse-graining transitions, (a) and (b) 
show coarse-graining rule 146 by rule 128. (a) shows results 
of running rule 146. The top line is the initial condition and 
time progress from top to bottom, (b) shows the results of 
running rule 128 with the coarse grained initial condition from 
(a), (c) and (d) show coarse-graining rule 105 by rule 150. (c) 
shows rule 105 and (d) shows rule 150. 



to limited computing power. Other transitions may exist 
with larger N. We observe that rule complexity never in- 
creases along the map's transitions, i.e., coarse-graining 
introduces (at least here) partial order among CA rules. 




2, 42 




10, 38, 46, 138 


171, 191 




139, 155, 174, 175 



FIG. 2: Coarse-graining transitions within the family of 256 
elementary CA. Only transitions with a cell block size N = 
2, 3, 4 are shown. An arrow indicates that the origin rules can 
be coarse-grained by the target rules and may correspond to 
several choices of N and P. 

As mentioned above, we were unable to coarse-grain 
16 elementary rules. 12 out of the 16 are the class 3 rules 
30,45,106 and their symmetries. The other four are the 
class 2 rule 154 and it's symmetries. We don't know if 
our inability to coarse-grain these 16 rules comes from 
limited computing power or from something deeper. 

It is worth noticing that a subset of the fixed points in 
the transition map is composed of all elementary additive 
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rules (see page 952 in Ref. §) and their symmetries. This 
result is not limited to elementary rules. All additive CA 
whose alphabet sizes S are prime numbers coarse-grain 
themselves with N — S and P the modulo S sum of 
S cells. We conjecture that there are situations where 
reducible fixed points exist for a wide range of systems, 
analogous to the emergence of amplitude equations in the 
vicinity of bifurcations points. 

Coarse-graining transitions can also exit the elemen- 
tary CA family. This happens whenever the alphabet of 
the coarse CA consists of more than two symbols. One 
such example which is of special importance is rule 110. 
Rule 110 is interesting because this class 4 CA is univer- 
sal in the Turing sense [22j and is therefore CIR 
It is capable of emulating all computations done by other 
computing devices in general and CA in particular. 

We found several ways to coarse-grain rule 110. Using 
N = 6, it is possible to project the 64 possible states onto 
an alphabet of 63 symbols. A more impressive reduction 
in the alphabet size is obtained by going to larger values 
of N. For N = 7, 8, 9, 10, 11 we found an alphabet reduc- 
tion of 6/128, 22/256, 67/512, 181/1024 and 463/2048 
respectively. We expect this behavior to persist for larger 
values of N. 

Another interesting coarse-graining of rule 110 that we 
found is the transition to rule 0. Rule has the trivial 
dynamics where all initial states evolve to the null con- 
figuration in a single time step. The transition to rule 
is possible because the cell combination "01010" is not 
generated by rule 110 and can only appear in the ini- 
tial state. Coarse-graining by rule is achieved using 
N = 5 and projecting "01010" to 1 and all other five cell 
combinations to 0. This example is important because 
it shows that even though rule 110 is CIR it has a pre- 
dictable coarse-grained dynamics (however trivial). To 
our knowledge rule 110 is the only proven CIR elemen- 



tary CA and therefore this is the only example of irre- 
ducible to reducible transition between elementary rules 
that we found. 

We did find other complex, potentially CIR rules that 
can be coarse-grained by reducible CA. Rules 18, 54, 126 
and their symmetries are coarse-grained by rule 0. As we 
showed above, rule 146 and it's symmetries can be coarse- 
grained by rule 128 in a non-trivial way. We don't know 
if these rules are CIR for lack of proof. Nevertheless, non- 
trivial irreducible to reducible transitions can in principle 
exist. Consider for example the CIR CA generated by the 
product of rules 110 and 128. Let the CA alphabet be 
{a, b} x {0, 1}, where the letters evolve according to rule 
110 and the digits according to 128. We can recover the 
reducible, coarse-grain-able rule 128 by projecting the 
alphabet onto {0, 1}. 

The fact that CIR rules can be coarse-grained and that 
they has predictable coarse-grained dynamics shows that 
CIR is not a good measure of physical complexity. As 
in the case of rule 110, a CIR system may still yield an 
efficient predictable theory, provided that we are willing 
to ask coarse-grained questions. It seems that a better 
classification of physical complexity is related to what 
classes of projection operator are required to coarse-grain 
the system: local, real space projections or more complex 
non-geometric projections? 

In summary, we have found that many CA, including 
CIR ones can be locally coarse-grained in space and time. 
In some cases CIR systems are predictable, if coarse- 
grained information only is required. 
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nautics and Space Administration through grant NAG8- 
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